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SIGNATURE EXTENSION IN REMOTE SENSING* 


C. B. CHITTINENI 

Lockheed Electronics Company, Inc. 

Systems and Services Division 
Houston, Texas 

ABSTRACT 

This paper considers the problem of signature extension in remote sensing. 
Signature extension is a process of increasing the spatial -temporal range 
over which a set of training statistics can be used to classify data without 
significant loss of recognition accuracy. 

Methods are developed for the selection of segments for obtaining the training 
data. Selection of the number of segments is treated as the problem of expan- 
sion of rectangular matrix with basis matrices. Computational algorithms 
based on mean minimum square estimation error are developed for the selection 
of best segments. Furthermore, a combinatorial algorithm for generating all 
possible r combinations of S in Sc r steps with a single change at each step 
is presented. 
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*The material for this paper was developed and prepared under Contract 
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1. INTRODUCTION 


In the application of remote sensing for large-area crop Inventories and 
other, the mul tispectral Land Satellite data are processed In units called 
segments (a segment Is an area 5 by 6 nautical miles). The processing of a 
segment necessitates the acquisition of labels of picture elements (pixels) 
to train the classifier. Obtaining the labels Is costly, and training the 
classifier for every segment is time consuming. 

To overcome the cost and time constraints, attempts have been made to solve 
the problem of signature extension; i.e., to train the classifier for classi- 
fying data acquired over large areas or many segments without significant 
loss of recognition accuracy. The goal of signature extension is then to 
minimize the requirements for obtaining the labels and extracting the training 
statistics. 

Many current signature extension techniques (refs. 1 to 3) are based on a 
transformation of training statistics to compensate for changes in Sun angle, 
atmospheric and viewing conditions, etc., between the training area and the 
recognition area. The signature extension transformation of these techniques 
is both multiplicative and additive. Minter (ref. 4) reviews the techniques 
proposed for signature extension in the literature. 

This paper considers an approach for signature extension (ref. 5) based on 
the assumptions that the data variations due to changes in Sun angle, 
atmospheric and viewing conditions, etc., can be significantly reduced by pre- 
processing and that the data are governed by a few inherent spectral 
classes related to the ground covers. With these assumptions, the training 
samples can be drawn from a few representative segments, and the classifier 
can be used to classify data acquired over large areas. 

Let S be the total number of segments. Suppo e that clustering these seg- 
ments produces T spectral classes with a total cf J blocks. The situation 
is illustrated in figure 1. 
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Figure 1.— Spectral clauses and blocks in measurement space. 


The number of spectral points counted in each block result in a matrix of 
segment versus count number for each block: 


A « 


-♦ Blocks 


1 2 ... J 


1 

2 


0 

0 


0 

0 


0 

0 


0 0 



Segments 


( 1 ) 


Based on matrix A, computational algorithms for the selection of training 
segments are developed. The segments selected are representative; i.e., 
all the blocks associated with the T spectral classes are present in the 
selected segments. Section 2 describes an orthonormal expansion for rectanu- 
lar matrices. Section 3 describes methods for the selection of best segments. 
Section 4 develops computational algorithms for the selection of individual 
segments. Section 5 presents a combinatorial algorithm for generating all 
possible r combinations out of S in Sc r steps with a single change at each 
step. Appendix presents a statistical description of the signature 
extension model, and appendix B derives matrix relations used in the paper. 

2. EXPANSION OF RECTANGULAR MATRIX WITH BASIS MATRICES 

In this section, the S * J rectangular matrix A is expanded in terms of 
basis matrices, and the basis matrices are obtained. From equation (1), 
form an S + J * S + J symmetric matrix B. 

0 A 

B - T (2) 

A 1 0 

Let C ar.d D be the eigenvector and eigenvalue matrices of B. It can easily 
be verified that 

BC = CD (3) 
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where 


L, 

1 

Jf 

1 , 

1 

.*♦ 

■ 71 




$ is an S * S orthonormal matrix, ^ Is a J * S matrix with orthonormal 
columns, and A is an S * S diagonal matrix; that Is, 

= (jxtT = i j 


* I ) 

Since C is an eigenvector matrix of a symmetric matrix, 

C T C = I 

equation (6) can easily be verified using equations (4a) and (5). Inner- 
multiplying both sides of equation (3) by matrix B of equation (2), 

BBC = BCD 


B 2 C » CD 2 

is obtained. Using equations (2), (4a), and (4b) in equation (7), 

K • IV 5<1 ft* 5*F ”■ 


• ■ ‘ILs ■ - k i ft* - !;*R" 


is obtained. Expanding equation (8) yields 

AA T <J> = 4> A 2 


A T Ai^ = <l> A 2 
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( 10 ) 


Let S < J. Since the maximum rank of A^A Is S, it will have at most S non- 
zero eigenvalues. Multiplying equation (9) on the left by A T and on the 
right by A* 1 gives 

A T A(A T <>A” 1 ) • (AW (11) 

On the comparison of equation (10) and (11), the following is obtained. 

» A T $A _1 (12) 

Taking the transpose of and innermul tiplying the result by A and $> yields 

A * <J>A^ (13) 

Expanding equation (13), matrix A can now be represented as 

S 

A c ^ X 1 <J> 1^1 O 4 ) 

i = l 

The following work shows that the relative importance of each term in equ*.- 

A 

tion (14) is proportional to X^. Let A be the approximation of A using 
t(<S) terms In equation (14). Define the squared error as 

e 2 * tr[(A - A) (A - A) T ] 


Equation (15) shows that if a term is dropped in equation (14), the representa- 
tion error in the mean square error sense is equal to the square of the 
corresponding eigenvalue. 

3. COMPUTATIONAL ALGORITHMS FOR THE SELECTION OF INDIVIDUAL SEGMENTS 


Based on the theory developed in the last section, a number m that gives an 
acceptable representation error can be chosen. This section considers the 
problem of choosing a particular set of m rows of matrix A or m segments and 
develops computational algorithms for their identification. 


3 . 1 MEAN SQUARE ERROR IN ESTIMATION 

Let the rows of matrix A be f j, 1 ■ 1 , 2 . . . , S. Arbitrarily let the 
first l segments be chosen. Let 


F t a C f l» f 2’ M 


(16) 


Where F, is a J * t rectangular matrix. Let the row 

T 

linear combination of the rows f ^ , i = 1 , 2, .... a. 


be estimated as a 
That Is, 



(17) 


where B is a vector of parameters. 
f, + j can be written as 



The estimation error between 



and 





(18) 


Differentiating equation (18) with respect to p and equating to zero yields 

2 

the 8, which minimizes Cj + - as 

6 = (19) 
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The substitution of equation (19) Into (18) results In the minimum error as 


c U ■ f lj[' • F »i F I F t>'’ F lKj 

■ tr {[' - F t( F I f ‘tl' ,F lK + J , I*j| UO) 

The mean minimum square error In the estimation of last (S - £) rows in terms 
of the first £ rows of matrix A can be written as 

c * (t) ' T5-TTT 

j«£+l 



1 

l/) 

I - F (F T F )"V^ 
1 r £' r £ r £' r £ 

- 

. i . V* f f T 

(s - £) Z-f j j 

j = £+l 

— 


3 . 2 SELECTION OF TH_E BEST m SEGMENTS 
2 

The quantity c (£) derived in section 3.1 measures the effectiveness of the 
selected £ rows in estimating the remaining S - £ rows. The computation can 
be organized in two ways in finding the best m rows. 


3.2.1 FORWARD SEQUENTIAL SEARCH 

The computation involves finding each additional row, one at a time. After 
selection of the r rows, the r + 1 th row is selected among the remaining 
S - r rows (by checking one at a time). This is a suboptimal procedure; it 
involves much less computation compared to an exhaustive search. Section 4 
gives the recursive expressions for reducing the computation. 

3.2.2 EXHAUSTIVE SEARCH 

A method of selecting m optimal rows for mean minimum square estimation 
error is accomplished by forming all possible Sc_ combinations and evalu- 

ty ' 

ating t (m) for each combination ani then selecting the best combination. 

2 

Section 4 gives recursive expressions for e (m) for reducing the amount of 
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computation. These results, coupled with the results of section 5, provides 
an efficient algorithm for the Implementation of exhaustive search In 
segment selection. 

4. RECURSIVE EXPRESSIONS FOR COMPUTATION OF MEAN 
MINIMUM SQUARE ESTIMATION ERROR 

In this section, recursive expressions are developed for the computation of 
mean minimum square estimation error when a row Is added to the selected 
segment set and when a row Is deleted from a selected segment set. 

4 . 1 CHAf'GE IN THE CRIT ERION WHEN A PA RTICULAR ROW JS ADDED 

Let fj , 1 « 1 , 2, . . . , r be the rows of matrix A selected at the rth 
step. From the rth step to the r ♦ ith step, a row that reduces the esti- 
mation error most Is added. Let be the row that Is added. Then, 

F r ■ [f,. f 2 f r ] (22) 

F rt) ■ [f,. f 2 f r . f r .,) ■ [F r f r .U (23) 


p 

In the rth step, the mean minimum square estimation error e (r) is 
expressed as 



8 


Consider 
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Now consider 


r*l( F r*l F rn) F r+1 " 


However, 


F r F r+l 


rA 


T 

r 

T 

r*1 


■ f rVl * f r.A f l * F r*r f I.l * “WJ.l 


wl ■ F r( B ;' ♦ B ;VfrM F I 


F r B ;' F r * 


C 2 * C 2 f r+l f r+l C 2 a 


F r+1 a r F r ' - f rO b r B ;' F r“ 


' - f rtlVl C 2 a 


(33) 


(34) 


(35) 


* f T r+ 1 f r+1 - <l } r r *~y r < r + } 

“ f r+l f r*l ‘ f r+l C 2 f r+l 


* tr ( f r+1 f r+l) * tr ( C 2 f r+l f r+l) 

From equations (33) through (36), 

F r*l(’>*l F r*l)' F r.l ■ c 2 * C 2 F r+l F r+l B 2 a ' W^1 C 2“ 


(36) 


' C 2 f r+1 f r+l a + af r+l f r+l 


(37) 
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Is obtained, where 



With equations (37), (38), and (39) in equation (25), (25) can be recursively 
computed from equation (24). 

4 . 2 CHANGE IN THE CRITERION WHEN A PARTICULAR ROW JS^LEJED 

Let fl, 1 ■ 1 , 2, . . . , r be the rows of matrix A selected at the rtk step. 

1 T 

Let a row f be deleted fr.,m this set. This section presents the expressions 
derived for the computation of mean minimum square estimation error with 
this reduced set. 

Let 



2 T 

The estimation error e (r - 1), with the selected rows f^ , 1 s 1 , 2, . . . , r - 1 , 
can be written as 



1 1 


Consider 




Let 



m 





r-l 




f T f 
r r 





The relationship between A f . and B ( , can be written as 


a a T 
r r 

a 


Consider 



■ M 


m 

A a„ 


I" f t 

r r 


V-l 

a T a 


f T 

r 


r 


o F ,A F T , ♦ f a T fJ. , + F_ ,a f T . ♦ 
r-l r r-l r r r-l r-l r r 


f . 
r r 


From equations (44) and (45), 






Is obtained. 






a 



F„ i a rV 
r-l r r 


- 

r r 


(42) 


(4J) 


(44) 


(45) 


(46) 
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Now consider 



With equations (43), (46), and (47) in equation (41), (41) can be recursively 
computed from equation (24). 


5. A COMBINATORIAL ALGORITHM FOR GENERATING ALL POSSIBLI COMBINATIONS 

This section describes an algorithm for generating all possible r combinations 
out of S in Sc r steps (ref. 6). At each step, a single change is made; i.e., 
one row is deleted and one is added. The recursive relations developed 
in section 4, coupled with this algorithm, can be used to search for r best 
segments out of all possible Sc^ combinations. 

The initial combination may be any combination in which all the r-selected 
numbers are consecutive. In the binary representation, it means that all 
the r l's are in one run in a vector of length S. For example, if r * 3 and 
S = 5, it may be started with 11100 or OHIO or 00111. The binary vector 
is denoted by A, and its i th component is A(i). Initially, all the components 
of A, except those of the last run, are marked. For example, if A * 00111000 
(for r * 3 and S * 8), then it is marked as 00111000. 


13 


If a Is a symbol, then a m stands for aa ... a, m times. Let 1 be the 
highest Index so that A(1) Is marked. A vector T(l), T(2), .... T(S) of 
Integers that satisfy the condition |T(j)| < j for j ■ 1, 2. .... S Is 
defined. Initially, T ( 1 ) * 0. If the Initial combination Is (&) p (T) r 0 S " r ‘ p , 
where S > r ♦ p, then T(p + r) * -1 and all the rest are immaterial. If 
the Initial combination is (tf)^' r 1 r , then T(S - r) * -1 and all the rest 
are immaterial. The changes that T must undergo in each combination gen- 
eration are described by subroutines a and 0 as follows: 

a: (1) If T(k) * 0, then output A and halt. 

(II) If T(k) > 0, then 1 «- T(k), output A, and go to step (1) of the 
procedure. 

(III) 1 «- k - 1. If T(k J > -(k - 1), then T(k - 1) ♦ T(k). 

(iv) Output A and go to step (1) of the procedure. 

0: (1) T(i ) «- -(k + 1). If T(k) > 0, then T(k + 1) - T(k), output A, 

and go to step (i) of the procedure. 

(11) T(k ♦ 1) «■ k - 1. If T(k) > -(k - 1), then T(k - 1) - T(k). 

(iii) Output A and go to step (1) of the procedure. 

Now the vector F(0), F(l), .... F(S) is introduced as follows. If A(m) * 1 
and it is the rightmost element in a run of Vs, then F(m) is the index 
of the first 1 of this run. If not, F(m) is immaterial. Let t be the index 
of the rightmost 1; that is, l = max m 

A(m)= 1 

An algorithm for generating all possible combinations with a single change 
at each step can now be described. The initial conditions of the algorithm 
are illustrated as follows. Let r = 3, S * 8 with an initial A = 01110000. 


Then 

i = 4. 



(1) 

k - i . If A(i) 

= 1 , go to step (8) . 


(2) 

j - F( O . 



(3) 

A ( i ) - 1, A ( j ) «- 

0. F(k) - k. If A(k 

- 1 ) = 1 and k > 1 , then 


F(k) - F(k - 1). 

FU)«-j + i;lfj< 

t, go to step (5). 
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(4) 

Z «- 1. 

Perform a. 




(5) 

If z < 

S, go to step (7). 




(6) 

1 - j. 

Perform 0. 




(7) 

1 «- z. 

Perform 0. 




(8) 

F(1 - 

1) «- F( 1 ) . If Z > 1, go to 

step (12). 


(9) 

A(1) - 

0, A(S) «■ 1, F(S) - S, Z «■ 

S. 

If 1 < s - 

1 , go to step (11 ) 

(10) 

Perform a. 




(ID 

1 «- S 

- 1 and perform 0. 




(12) 

J «■ F(i) 




(13) 

A(1) - 

0, A( j - 1) - 1, F(t) - j ■ 

- 1. 

If z < s. 

go to step (17). 

(14) 

If Z + 

W j - 1 , go to step (16). 




(15) 

Perform a. 




(16) 

1 - j 

- 2. Perform 0. 




(17) 

i * z. 

Perform 0. 
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APPENDIX A 


STATISTICAL INTERPRETATION OF THE SIGNATURE EXTENSION MODEL 

In this appendix, a statistical interpretation of the signature extension 
model considered In the paper is given. Let 1, t, and j respectively be 
the segment Index, the spectral class index, and the block index. 

Let h( x | L 3 t) be the density function of the patterns in the tth spectral 
class. Let f(x|s 3 i ) be the density function of the patterns in the seg- 
ments. Let T be the total number of spectral classes, J the maximum number 
of blocks, and S the total number of segments. Consider 

T 

f ( x | s * 1) 1 ^ f(x, L » t|s 3 i) 
t 3 l 

T 

= ^ ' f ( x | L = t, s = i)P(L = t | s = 1) 
t*l 

T 

s Y. h(x|L 3 t)P(L - 1 1 S * i) (A-l ) 

t=l 

where it is assumed that h(x|L = t) = f ( x | L 3 t, s 3 i). Let the euclidean 
space be partitioned into a set of blocks B 1 , f^, .... and let 

P(xrB.|L 3 t) = /h(x|L 3 t)dx 
J B . 


A-l 


and 


P(xrB . | s • 1) * fn x | s ■ 1) 

J b j 


dx 


» yi /*P(L ■ 1 1 s ■ 1 ) h( x | L « t)dx 
t«l 

T 

5 P ( L * t|s * i ) P(xeBj | L * t) 

t«l 


(A-2) 


Introducing a matrix notation, one can write equation (A-2) as 
1, 2, •••, J blocks 1, 2, ••*, T spectral classes 

1 
2 


S 1 
segments 


1 , 2, • • • , J blocks 
J< 


9 m 

1 


“ 

1 . 



2 



2 



• • 



\ 



• 



* t 


_ 

1 S i | 


T 

_ _ 


seoments 
P = AB 


spectral 

classes 


(A-3) 


where P is an S * J matrix with elements p.j = P(xeBj|s ■ i), A is an S * T 
matrix with elements a it * P(L = t|s = i), and B is an T x J matrix with 
elements b t j * P(xeBj|l * t). Equation (A-3) describes the probabilistic 
relationship among segments, spectral classes, and blocks. 
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APPENDIX B 


MATRIX RELATIONSHIPS 

This appendix derives the matrix relationships used in section 4. 
and B be the Inverse matrices of each other and are as shown below: 



* A r 


■B r »; 

A > 

.1 a 

and B * 

b! 6 


L r J 


r M J 


Since A Is the Inverse of B, BA * I. That Is 


BA 


B b 
r r 

L »; ». 


A r a r 
a r a . 


Expansion of equation (B-2) gives 


BA + b a ■ I 
r r r r 


B r a r + <ib r » 0 


b^A r + BaJ = 0 


b r a r + ot8 * 1 


From equation (B-3), 


(B/r) - ' ■ a ;’ b ;’ * (■ - vl )' 1 


i + 


ba T 
r r 


1 - a r b r 

r r 


is obtained. From equations (B-5), (B-6), and (B-7), 

T 


B' 


•1 


A r a 


Vr 


B-l 


Let A 
(B-l) 

(B-2) 

(B-3) 

(B-4) 

(B-5) 

(B-6) 

(B-7) 

(B-8) 


Is obtained. Equation (B-8) Is used In section 4.2. From equation (B-4), 

a r » -B; ] b r a (B-9) 

Is obtained. Substitution of equation (B-9) into (B-6) yields 

a « 1-t— (B-10) 

B - b\\ 

From equations (B-3) and (B-9), 

A r « B* 1 + B'VbV’a (B-ll ) 

r r r r r r 

is derived. Equations (B-9), B-10), and (B-ll) are used In section 4.1. 
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